Control sequence generation system and methods

ABSTRACT

A model receives a target demand curve as an input and outputs an optimized control sequence that allows equipment within a physical space to be run optimally. A thermodynamic model is created that represents equipment within the physical space, with the equipment being laid out as nodes within the model according to the equipment flow in the physical space. The equipment activation functions comprise equations that mimic equipment operation. Values flow between the nodes similarly to how states flow between the actual equipment. The model is run such that a control sequence is used as input into the neural network; the neural network outputs a demand curve which is then checked against the target demand curve. Machine learning methods are then used to determine a new control sequence. The model is run until a goal state is reached.

RELATED APPLICATION

The present application hereby incorporates by reference the entiretyof, and claims priority to, U.S. provisional patent application Ser. No.62/704,976 filed Jun. 5, 2020.

The present application hereby incorporates by reference U.S. utilitypatent application Ser. No. 17/009,713, filed Sep. 1, 2020.

FIELD

The present disclosure relates to neural network methods for determiningbuilding state needs. More specifically the present disclosure discussesgenerating control sequences when given zone state loads.

BACKGROUND

Structures need to have their states modified to meet the needs of thepeople or objects in the structure. For example, instruments may need tohave specific humidity and temperature levels to avoid damage, and weall know how miserable it is to be in a too-hot or too-cold building.However, it is a daunting problem to attempt to keep a building at adesired state or move it to a new one due to the complex interactionbetween the building and the outside world, the people in the building,and the uneven heating (as a state example) provided by equipment.

SUMMARY

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription section. This summary does not identify required oressential features of the claimed subject matter.

In embodiments, a control sequence generation system implemented by oneor more computers is disclosed, comprising: a time series controlsequence that comprises instructions to control the controllable deviceover time; a target demand curve that comprises amount of state in alocation over time; a thermodynamic model comprised of a thermodynamicrepresentation of at least one controllable device; a computing enginethat runs the thermodynamic model using the time series control sequenceas input, and outputs a simulated demand curve; a cost functiondeterminer that compares difference between the simulated demand curveand the target demand curve producing a cost; a new model inputdeterminer that uses the cost to create a next time series controlsequence; an iterator, which iteratively runs the computing engine usingthe next time series control sequence as input, the cost functiondeterminer and the new model input determiner until a stop state isreached; wherein when the stop state is reached, the next time seriescontrol sequence becomes the control sequence generated.

In embodiments, the cost function determiner comprises a cost functionwhich comprises a next time series control sequence of operation error,a next time series output path error, an energy cost, a state cost, ashort cycling frequency, a frequency of sequence changes, a equipmentlife cost, or a comfort value.

In embodiments, the thermodynamic model is a heterogenous physicsnetwork.

In embodiments, the heterogenous physics network comprises nodes thatcomprise equations that model thermodynamic behavior of equipment.

In embodiments, the nodes are laid out with reference to the physicalequipment behavior.

In embodiments, at least one node has multiple variables associated withit.

In embodiments, at least two nodes have different activation functions.

In embodiments, at least one activation function comprises multipleequations.

In embodiments, the new model input determiner further comprises abackpropagator that determines a gradient of values in nodes of thethermodynamic model in relation to the cost.

In embodiments, the backpropagator taking the gradient of thethermodynamic model backward using automatic differentiation.

In embodiments, the new model input determiner further comprises anoptimizer that uses the gradient of values to produce a next time seriescontrol sequence; the control sequence generated is used to control atleast one controllable device.

In embodiments, a node within the thermodynamic model accepts a timeseries curve representing weather as input.

In embodiments, a method of control sequence generation implemented byone or more computers is disclosed, comprising: receiving a neuralnetwork of a plurality of controlled devices; receiving a desired demandcurve; receiving a simulated control sequence that comprisesinstructions to control the controllable device over time for at leastone of the plurality of controlled building zones; performing a machinelearning process to run the neural network using a simulated controlsequence as input and receiving a simulated demand curve as output;computing a cost function using the simulated demand curve and thedesired demand curve; using the cost function to determine a newsimulated control sequence; iteratively executing the performing,computing, and using steps until a goal state is reached; anddetermining that the new simulated control sequence is the time seriescontrol sequence upon the goal state being reached.

In embodiments, the demand curve is a time series of zone energy inputs.

In embodiments, computing the cost function further comprisesdetermining difference between the desired demand curve and thesimulated demand curve.

In embodiments, performing the machine learning process furthercomprises performing automatic differentiation backward through theneural network producing a new time series control sequence.

In embodiments, the goal state comprises the cost function beingminimized, the neural network running for a specific time, or the neuralnetwork running a specific number of iterations.

In embodiments, the neural network comprises multiple activationfunctions within its nodes and wherein a multiple activation functionhas multiple variables that are passed between nodes.

In embodiments, a computer-readable storage medium configured withexecutable instructions to perform a method for creation of a demandcurve upon receipt of a comfort curve, is disclosed, the methodcomprising: receiving a neural network of a plurality of controlleddevices; receiving a desired demand curve; receiving a time seriescontrol sequence that comprises instructions to control the controllabledevice over time for at least one of the plurality of controlledbuilding zones; performing a machine learning process to run the neuralnetwork using a simulated time series control sequence as input andreceiving a simulated demand curve as output; computing a cost functionusing the simulated demand curve and the desired demand curve; using thecost function to determine a new simulated time series control sequence;iteratively executing the performing, computing, and using steps until agoal state is reached; and determining that the new simulated timeseries control sequence is the time series control sequence upon thegoal state being reached.

Additional features and advantages will become apparent from thefollowing detailed description of illustrated embodiments, whichproceeds with reference to accompanying drawings.

BRIEF DESCRIPTION OF THE FIGURES

Non-limiting and non-exhaustive embodiments of the present embodimentsare described with reference to the following FIGURES, wherein likereference numerals refer to like parts throughout the various viewsunless otherwise specified.

FIG. 1 is a functional block diagram showing an exemplary embodiment ofa control sequence generation system in conjunction with which describedembodiments can be implemented.

FIG. 2 is a block diagram of an exemplary computing environment inconjunction with which described embodiments can be implemented.

FIG. 3 depicts a system that determines control sequence curves fromdemand curves with which described embodiments can be implemented.

FIG. 4 is a flow diagram showing an exemplary embodiment of a method todetermine control sequences from demand curves with which describedembodiments can be implemented.

FIG. 5 is a diagram showing a high level exemplary embodiment of theinput and output of a neural network model with which describedembodiments can be implemented.

FIG. 6 is a functional block diagram showing different machine learningfunctions with which described embodiments can be implemented.

FIG. 7 depicts a physical system whose behavior can be determined byusing a neural network with which described embodiments can beimplemented.

FIG. 8 depicts a neural network that may be used to model behaviors ofthe physical system of FIG. 7 with which described embodiments can beimplemented.

FIG. 9 is a block diagram showing an exemplary embodiment of a system todetermine control sequences from demand curves with which describedembodiments can be implemented.

FIG. 10 depicts a node system that has multiple inputs with whichdescribed embodiments can be implemented.

Corresponding reference characters indicate corresponding componentsthroughout the several views of the drawings. Skilled artisans willappreciate that elements in the FIGURES are illustrated for simplicityand clarity and have not necessarily been drawn to scale. For example,the dimensions of some of the elements in the figures may be exaggeratedrelative to other elements to help to improve understanding of variousembodiments. Also, common but well-understood elements that are usefulor necessary in a commercially feasible embodiment are often notdepicted in order to facilitate a less obstructed view of these variousembodiments.

DETAILED DESCRIPTION

Disclosed below are representative embodiments of methods,computer-readable media, and systems having particular applicability tosystems and methods for building neural networks that describe physicalstructures. Described embodiments implement one or more of the describedtechnologies.

Various alternatives to the implementations described herein arepossible. For example, embodiments described with reference to flowchartdiagrams can be altered, such as, for example, by changing the orderingof stages shown in the flowcharts, or by repeating or omitting certainstages.

“Optimize” means to improve, not necessarily to perfect. For example, itmay be possible to make further improvements in a value or an algorithmwhich has been optimized.

“Determine” means to get a good idea of, not necessarily to achieve theexact value. For example, it may be possible to make furtherimprovements in a value or algorithm which has already been determined.

A “cost function,” generally, compares the output of a simulation modelwith the ground truth—a time curve that represents the answer the modelis attempting to match. This gives us the cost—the difference betweensimulated truth curve values and the expected values (the ground truth).The cost function may use a least squares function, a Mean Error (ME),Mean Squared Error (MSE), Mean Absolute Error (MAE), a Categorical CrossEntropy Cost Function, a Binary Cross Entropy Cost Function, and so on,to arrive at the answer. In some implementations, the cost function is aloss function. In some implementations, the cost function is athreshold, which may be a single number that indicates the simulatedtruth curve is close enough to the ground truth. In otherimplementations, the cost function may be a slope. The slope may alsoindicate that the simulated truth curve and the ground truth are ofsufficient closeness. When a cost function is used, it may be timevariant. It also may be linked to factors such as user preference, orchanges in the physical model. The cost function applied to thesimulation engine may comprise models of any one or more of thefollowing: energy use, primary energy use, energy monetary cost, humancomfort, the safety of building or building contents, the durability ofbuilding or building contents, microorganism growth potential, systemequipment durability, system equipment longevity, environmental impact,and/or energy use CO2 potential. The cost function may utilize adiscount function based on discounted future value of a cost. In someembodiments, the discount function may devalue future energy as comparedto current energy such that future uncertainty is accounted for, toensure optimized operation over time. The discount function may devaluethe future cost function of the control regimes, based on the accuracyor probability of the predicted weather data and/or on the value of theenergy source on a utility pricing schedule, or the like.

A “goal state” may read in a cost (a value from a cost function) anddetermine if that cost meets criteria such that a goal has been reached.Such criteria may be the cost reaching a certain value, being higher orlower than a certain value, being between two values, etc. A goal statemay also look at the time spent running the simulation model overall, ifa running time has been reached, the neural network running a specificnumber of iterations, and so on.

A machine learning process is one of a variety of computer algorithmsthat improve automatically through experience. Common machine learningprocesses are Linear Regression, Logistic Regression, Decision Tree,Support Vector Machine (SVM), Naive Bayes, K-Nearest Neighbors (kNN),K-Means Clustering, Random Forest, Backpropagation with optimization,etc.

An “optimization method” is a method that takes a reverse gradient of acost function with respect to an input of a neural network, anddetermines an input that more fully satisfies the cost function; thatis, the new input leads to a lower cost, etc. Such optimization methodsmay include gradient descent, stochastic gradient descent, min-batchgradient descent, methods based on Newton's method, inversions of theHessian using conjugate gradient techniques, Evolutionary computationsuch as Swarm Intelligence, Bee Colony optimization; SOMA, and ParticleSwarm, etc. Non-linear optimization techniques, and other methods knownby those of skill in the art may also be used.

In some machine learning techniques, backpropagation may be performed byautomatic differentiation, or by a different method to determine partialderivatives of the node values within a neural network.

A “state” as used herein may be Air Temperature, Radiant Temperature,Atmospheric Pressure, Sound Pressure, Occupancy Amount, Indoor AirQuality, CO2 concentration, Light Intensity, or another state that canbe measured and controlled.

I. Overview

Artificial neural networks are powerful tools that have changed thenature of the world around us, leading to breakthroughs inclassification problems, such as image and object recognition, voicegeneration and recognition, autonomous vehicle creation and new medicaltechnologies, to name just a few. However, neural networks start fromground zero with no training. Training itself can be very onerous, bothin that an appropriate training set must be assembled, and that thetraining often takes a very long time. For example, a neural network canbe trained for human faces, but if the training set is not perfectlybalanced between the many types of faces that exist, even afterextensive training, it may still fail for a specific subset; at best,the answer is probabilistic; with the highest probability beingconsidered the answer.

Existing approaches offer three steps to develop a deep learning AImodel. The first step builds the structure of a neural network throughdefining the number of layers, number of nodes in each layer, anddetermines the activation function that will be used for the neuralnetwork. The second step determines what training data will work for thegiven problem, and locates such training data. The third step attemptsto optimize the structure of the model, using the training data, throughchecking the difference between the output of the neural network and thedesired output. The network then uses an iterative procedure todetermine how to adjust the weights to more closely approach the desiredoutput. Exploiting this methodology is cumbersome, at least becausetraining the model is laborious.

Once the neural network is trained, it is basically a black box,composed of input, output, and hidden layers. The hidden layers are welland truly hidden, with no information that can be gleaned from themoutside of the neural network itself. Thus, to answer a slightlydifferent question, a new neural network, with a new training set mustbe developed, and all the computing power and time that is required totrain a neural network must be employed.

We describe herein a way to use a neural network to determine optimalcontrol states for equipment (on, off, running at some intermediatevalue) within a physical space when given the energy input amountsneeded for various zones within the state—zone energy inputs, or demandcurves. “Physical space” should be understood broadly—it can be abuilding, several buildings, buildings and grounds around it, a definedoutside space, such as a garden or an irrigated field, etc. A portion ofa building may be used as well. For example, a floor of a building maybe used, a random section of a building, a room in a building, etc. Thismay be a space that currently exists, or may be a space that exists onlyas a design. Other choices are possible as well.

The physical space may be divided into zones. Different zones may havedifferent sets of requirements for the amount of state needed in thezone to achieve the desired values. For example, for the state“temperature,” a user Chris may like their office at 72° from 8 am-5 pm,while a user Avery may prefer their office at 77° from 6 am-4 pm. Thesepreferences can be turned into comfort curves, which are a chronological(time-based) state curve. Chris's office comfort curve may be 68° fromMidnight to 8 am, 72° from 8 am to 5 pm, then 68° from 5 pm to midnight.The comfort curves (for a designated space, such as Chris's office), arethen used to calculate demand curves, which are the amount of state thatmay be input into the associated zones to achieve the state desired overtime. For Chris's office, that is the amount of heat (or cold) that maybe pumped into their office for the 24 hour time period covered by thecomfort curve, that is, a zone energy input. These zones are controlledby one or more equipment pieces, allowing state in the space to bechanged. Such zones may be referred to as controlled building zones.

Once we have one or more demand curves, we then run an equipment neuralnetwork model forward with a control sequence as input to determinedemand output for that demand curve. That is, when we run equipment atvarious times in a structure, the structure, in turn, has some amount ofstate over time. This can be thought of as running a furnace from time Tto time T+20, and from time T+120 to time T+145 in a structure with twozones. The heat propagates through both zones in the neural networkwhich includes the walls, the air, etc, and diffuses. The model outputsthe amount of heat, in this case, from time T to time T+240 in bothzones, giving us two demand curves. We then check the demand curveoutput with the desired “ground truth” demand curve using a costfunction, and then machine learning curves are used to tune the inputvalues to create a new control sequence (or sequences). In someembodiments, a gradient of the cost function is calculated throughbackpropagation to the input (e.g, a resource), and then optimized by,e.g., a type of gradient descent, etc., giving us a new control curve totry. This is repeated until a goal state is reached. The last controlsequence run is the control sequence that is then used to determineoptimal equipment operation.

The neural networks disclosed herein have potentially differentactivation functions that may be equations that model differentresources. For example, a pump will be described by different equationsthan a motor, a boiler, a heating coil, etc.

II. Computing Environment

FIG. 1 illustrates a generalized example of a suitable computingenvironment 100 in which described embodiments may be implemented. Thecomputing environment 100 is not intended to suggest any limitation asto scope of use or functionality of the disclosure, as the presentdisclosure may be implemented in diverse general-purpose orspecial-purpose computing environments.

With reference to FIG. 1 , the core processing is indicated by the coreprocessing 130 box. The core processing 130 includes at least onecentral processing unit 110 and memory 120. The central processing unit110 executes computer-executable instructions and may be a real or avirtual processor. It may also comprise a vector processor 112, whichallows same-length node strings to be processed rapidly. In amulti-processing system, multiple processing units executecomputer-executable instructions to increase processing power and assuch the vector processor 112, GPU 115, and CPU can be runningsimultaneously. The memory 120 may be volatile memory (e.g., registers,cache, RAM), non-volatile memory (e.g., ROM, EEPROM, flash memory,etc.), or some combination of the two. The memory 120 stores software185 implementing the described methods of generating control sequences.

A computing environment may have additional features. For example, thecomputing environment 100 includes storage 140, one or more inputdevices 150, one or more output devices 155, one or more networkconnections (e.g., wired, wireless, etc.) 160 as well as othercommunication connections 170. An interconnection mechanism (not shown)such as a bus, controller, or network interconnects the components ofthe computing environment 100. Typically, operating system software (notshown) provides an operating environment for other software executing inthe computing environment 100, and coordinates activities of thecomponents of the computing environment 100. The computing system mayalso be distributed; running portions of the control sequence generationsoftware 185 on different CPUs.

Embodiments may also be implemented in cloud computing environments. Inthis description and the following claims, “cloud computing” may bedefined as a model for enabling ubiquitous, convenient, on-demandnetwork access to a shared pool of configurable computing resources(e.g., networks, servers, storage, applications, and services) that canbe rapidly provisioned via virtualization and released with minimalmanagement effort or service provider interaction, and then scaledaccordingly. A cloud model can be composed of various characteristics(e.g., on-demand self-service, broad network access, resource pooling,rapid elasticity, measured service, etc.), service models (e.g.,Software as a Service (“SaaS”), Platform as a Service (“PaaS”),Infrastructure as a Service (“IaaS”), and deployment models (e.g.,private cloud, community cloud, public cloud, hybrid cloud, etc.).

The storage 140 may be removable or non-removable, and includes magneticdisks, magnetic tapes or cassettes, CD-ROMs, CD-RWs, DVDs, flash drives,or any other medium which can be used to store information and which canbe accessed within the computing environment 100. The storage 140 storesinstructions for the software, such as control sequence generationsoftware 185 to implement methods of node discretization and creation.

The input device(s) 150 may be a device that allows a user or anotherdevice to communicate with the computing environment 100, such as atouch input device such as a keyboard, video camera, a microphone,mouse, pen, or trackball, and a scanning device, touchscreen, or anotherdevice that provides input to the computing environment 100. For audio,the input device(s) 150 may be a sound card or similar device thataccepts audio input in analog or digital form, or a CD-ROM reader thatprovides audio samples to the computing environment. The outputdevice(s) 155 may be a display, printer, speaker, CD-writer, or anotherdevice that provides output from the computing environment 100.

The communication connection(s) 170 enable communication over acommunication medium to another computing entity. The communicationmedium conveys information such as computer-executable instructions,compressed graphics information, or other data in a modulated datasignal. Communication connections 170 may comprise input devices 150,output devices 155, and input/output devices that allows a client deviceto communicate with another device over network 160. A communicationdevice may include one or more wireless transceivers for performingwireless communication and/or one or more communication ports forperforming wired communication. These connections may include networkconnections, which may be a wired or wireless network such as theInternet, an intranet, a LAN, a WAN, a cellular network or another typeof network. It will be understood that network 160 may be a combinationof multiple different kinds of wired or wireless networks. The network160 may be a distributed network, with multiple computers, which mightbe building controllers, acting in tandem. A computing connection 170may be a portable communications device such as a wireless handhelddevice, a cell phone device, and so on.

Computer-readable media are any available non-transient tangible mediathat can be accessed within a computing environment. By way of example,and not limitation, with the computing environment 100,computer-readable media include memory 120, storage 140, communicationmedia, and combinations of any of the above. Computer readable storagemedia 165 which may be used to store computer readable media comprisesinstructions 175 and data 180. Data Sources may be computing devices,such as general hardware platform servers configured to receive andtransmit information over the communications connections 170. Thecomputing environment 100 may be an electrical controller that isdirectly connected to various resources, such as HVAC resources, andwhich has CPU 110, a GPU 115, Memory, 120, input devices 150,communication connections 170, and/or other features shown in thecomputing environment 100. The computing environment 100 may be a seriesof distributed computers. βThese distributed computers may comprise aseries of connected electrical controllers.

Although the operations of some of the disclosed methods are describedin a particular, sequential order for convenient presentation, it shouldbe understood that this manner of description encompasses rearrangement,unless a particular ordering is required by specific language set forthbelow. For example, operations described sequentially can be rearrangedor performed concurrently. Moreover, for the sake of simplicity, theattached figures may not show the various ways in which the disclosedmethods, apparatus, and systems can be used in conjunction with othermethods, apparatus, and systems. Additionally, the description sometimesuses terms like “determine,” “build,” and “identify” to describe thedisclosed technology. These terms are high-level abstractions of theactual operations that are performed. The actual operations thatcorrespond to these terms will vary depending on the particularimplementation and are readily discernible by one of ordinary skill inthe art.

Further, data produced from any of the disclosed methods can be created,updated, or stored on tangible computer-readable media (e.g., tangiblecomputer-readable media, such as one or more CDs, volatile memorycomponents (such as DRAM or SRAM), or nonvolatile memory components(such as hard drives) using a variety of different data structures orformats. Such data can be created or updated at a local computer or overa network (e.g., by a server computer), or stored and accessed in acloud computing environment.

FIG. 2 depicts a distributed computing system with which embodimentsdisclosed herein may be implemented. Two or more computerizedcontrollers 205 may comprise all or part of a computing environment 100,210. These computerized controllers 205 may be connected 215 to eachother using wired or wireless connections 215. These computerizedcontrollers may comprise a distributed system that can run without usingconnections (such as internet connections) outside of the computingsystem 200 itself. This allows the system to run with low latency, andwith other benefits of edge computing systems.

FIG. 3 discloses a system 300 that determines control sequence curvesfrom demand curves. In an exemplary environment, a control sequencegeneration system comprises inputting a demand curve 305 (that is, stateneeds, such as desired temperature over time) into a neural networkequipment model 315. Using machine learning techniques, the equipmentmodel 315 produces a control state sequence 310 that gives controlsequences that can then be used by controllable equipment 320. This mayproduce the desired amount of state (as represented by the originaldemand curve) in a given space.

III. Exemplary Method Embodiment

FIG. 4 depicts one method 400 for control sequence generation. Theoperations of method 400 presented below are intended to beillustrative. In some embodiments, method 400 may be accomplished withone or more additional operations not described, and/or without one ormore of the operations discussed. Additionally, the order in which theoperations of method 400 are illustrated in FIG. 4 and described belowis not intended to be limiting. In some embodiments, method 400 may beimplemented in one or more processing devices (e.g., a distributedsystem, a digital processor, an analog processor, a digital circuitdesigned to process information, an analog circuit designed to processinformation, a state machine, and/or other mechanisms for electronicallyprocessing information). The one or more processing devices may includeone or more devices executing some or all of the operations of method400 in response to instructions stored electronically on an electronicstorage medium. The one or more processing devices may include one ormore devices configured through hardware, firmware, and/or software tobe specifically designed for execution of one or more of the operationsof method 400.

At operation 405, a neural network of a plurality of controlled devicesis received. The neural network model may have been stored in memory,and so may be received from the processing device that the model isbeing run on. In some implementations, the neural network model may bestored within a distributed system, and received from more than oneprocessor within the distributed system, etc. A controlled device is adevice that has controls, such as on-off switches, motors, variablecontrols, etc. such that a computer can modify its behavior. Thesecontrols may be wired, wireless, etc.

In some embodiments described herein, in a neural network, thefundamentals of physics are utilized to model single components orpieces of equipment on a one-to-one basis with neural network nodes.Some nodes use physics equations as activation functions. Differenttypes of nodes may have different equations for their activationfunctions, such that a neural network may have multiple activationfunctions within its nodes. When multiple components are linked to eachother in a schematic diagram, a neural network is created that modelsthe components as nodes. The values between the objects flow between thenodes as weights of connected edges. These neural networks may model notonly the real complexities of systems but also their emergent behaviorand the system semantics. Therefore, they may bypass two major steps ofthe conventional AI modeling approaches: determining the shape of theneural net, and training the neural network from scratch.

As the nodes are arranged in order of an actual system (or set ofequations) and because the nodes themselves comprise an equation or aseries of equations that describe the function of their associatedobject, and certain relationships between them are determined by theirlocation in the neural net, a huge portion of training is no longernecessary, as the neural network itself comprises location information,behavior information, and interaction information between the differentobjects represented by the nodes. Further, the values held by nodes inthe neural network at given times represent real-world behavior of theobjects so represented. The neural network is no longer a black box butitself contains important information. This neural network structurealso provides much deeper information about the systems and objectsbeing described. Since the neural network is physics- andlocation-based, unlike the conventional AI structures, it is not limitedto a specific model, but can run multiple models for the system that theneural network represents without requiring separate creation ortraining.

In some embodiments, the neural network that is described herein choosesthe location of the nodes to tell you something about the physicalnature of the system. The nodes are arranged in a way that referencesthe locations of actual objects in the real work. The neural networkalso may use actual equations that can be used to determine objectbehavior into the activation function of the node. The weights that movebetween nodes are equation variables. Different nodes may have unrelatedactivation functions, depending on the nature of the model beingrepresented. In an exemplary embodiment, each activation function in aneural network may be different.

As an exemplary embodiment, a pump could be represented in a neuralnetwork as a network node with multiple variables (weights on edges),some variables that represent efficiency, energy consumption, pressure,etc. The nodes will be placed such that one set of weights (variables)feeds into the next node (e.g., with equation(s) as its activationfunction) that uses those variables. Unlike other types of neuralnetworks, two required steps in earlier neural network versions—shapingthe neural net, and training the model—may already be performed. Usingembodiments discussed herein the neural network model need not betrained on some subset of information that is already known. In someembodiments, the individual nodes represent physical representations.Individual nodes may hold parameter values that help define the physicalrepresentation. As such, when the neural network is run, the parametershelping define the physical representation can be tweaked to moreaccurately represent the given physical representation.

This has the effect of pre-training the model with a qualitative set ofguarantees, as the physics equations that describe objects being modeledare true, which saves having to find training sets and using hugeamounts of computational time to run the training sets through themodels to train them. A model does not need to be trained withinformation about the world that is already known. With objectsconnected in the neural network similar to how they are connected in thereal world, emergent behavior arises in the model that, in certaincases, maps to the real world. This model behavior that is uncovered isoften otherwise too computationally complex to determine. Further, thenodes represent actual objects, not just black boxes. The behavior ofthe nodes themselves can be examined to determine behavior of theobject, and can also be used to refine the understanding of the objectbehavior. One example of such heterogenous models is described in U.S.patent application Ser. No. 17/143,796, filed on Jan. 7, 2021, which isincorporated herein in its entirety by reference.

At operation 410, a simulated control sequence is received. Initially,the values of the control sequence curve may be random, may be a controlsequence from another similar model run, etc. The control curvecomprises instructions to control one or more controllable devices overtime for at least one of the plurality of controlled building zones thatare modeled by the neural network. As a brief overview, in anillustrative embodiment, we have the demand curves we want zones (e.g.,areas) to conform to, such as Chris's office, as described above, and wewish to find the control sequences (i.e., how to run the equipment) timeto meet the state amount indicated by the demand curve. To do so, we usesimulated control curves that control (simulated) equipment by turningit on, off, and set them to intermediate values, etc. as input in themodel, run the model which outputs the simulated comfort curve for thegiven demand curve.

FIG. 5 is a diagram 500 showing an high level exemplary embodiment ofthe input and output of a neural network model. With reference to FIG. 5, this entails using a control sequence (e.g., equipment behavior overtime t0 to t24) as input into an equipment model 515. The equipmentmodel runs forward and produces a simulated demand curve 510 for thesame time period (t0 to t24) as the control sequence. A new controlsequence is determined and then fed back 520 into the model 515, until asuitable simulated demand curve is created. This entails the simulateddemand curve being sufficiently close to the desired demand curve.

At operation 420, a machine learning process is performed to run theneural network using a simulated control sequence as input and receivingthe simulated demand curve 425 as output. Running the model may entailfeedforward—running the control sequence though the model to the outputsover time T(0)-T(n), capturing state output values—within neurons thatrepresent resources that modify state—over the same time T(0)-T(n). Atoperation 425, simulated demand curve(s) are output. In someembodiments, the demand curve is output 425 successively in timestepsduring the model run, or other methods are used. The first time theneural network is run, a control sequence 410 may be supplied. Thisinitial control sequence may be determined randomly, or another methodmay be used, such as a control sequence stored previously that was usedas the solution to a similar demand curve problem.

At operation 415, the desired demand curve(s) are received. These arethe curves that describe the amount of state that is needed over time.These may also be called ground truth demand curves. Ground truth is theinformation provided by direct evidence or is the desired output fromthe neural network.

At operation 430, a cost function is computed using the time series ofdesired comfort curve(s) and the model output—a simulated demand curve.The cost function measures the difference between the time series ofdesired demand curve(s) 415 and the simulated demand curve(s) output 425from the neural network 420. Details of the cost function are describedelsewhere.

At operation 435, a goal state is checked to determine if a stoppingstate has been reached. The goal state may be that the cost from thecost function is within a certain value, that the program has run for agiven time, that the model has run for a given number of iterations,that the model has run for a given time, that a threshold value has beenreached, such as the cost function should be equal or lower than thethreshold value, or a different criterion may be used. If the goal statehas not been reached, then a new set of inputs needs to be determinedthat are incrementally closer to an eventual answer—a lowest (or highestor otherwise determined) value for the cost function, as describedelsewhere.

At operation 445, if the goal state 435 has determined that a stoppingstate been reached, then the control sequence that was used for the lastheterogenous model run is set as the solved control sequence; that is,the control sequence that will meet the requirements for the desireddemand curve, within some range. This method can save as much as 30% ofenergy costs over adjusting the state when the need arises. If the goalstate has not been reached, then the determine new control sequence step440, the run neural network step 420, the output simulation demand curvestep 425, and compute cost function state 430 are iteratively performed,which incrementally optimizes the demand curve until the goal state 435is reached.

In some implementations, once the control sequence has been determined445, it can then be used to run the equipment that is associated withthe equipment modeled in the neural network. Controlling equipment insuch a predetermined fashion can save greatly on energy costs, as wellas more accurately controlling state in a defined space for the peopleand objects therein.

At operation 440 new simulated control sequences are determined for thenext run of the neural network. This may be performed using the costfunction; by using machine learning algorithms, etc. In someembodiments, backpropagation is used to determine a gradient of the costfunction in relation to the various values in the neural network. Thisgradient may then be used to optimize the control sequence for the nextmodel run.

FIG. 6 is a functional block diagram 600 showing different machinelearning functions. At operation 440, control sequences are determined.These control sequences may be determined by using machine learning 605.Machine learning techniques 605 may comprise determining gradients ofthe various variables within the neural network with respect to the costfunction. Once the gradients are determined, gradient methods may beused to incrementally optimize the control sequences. The gradient at alocation shows which way to move to minimize the cost function withrespect to the inputs—i.e., the control sequences. In some embodiments,gradients of the internal variables with respect to the cost functionare determined 610. In some embodiments, internal parameters of thenodes have their partial derivatives calculated, which gives thegradient. Different nodes may have different parameters. For example, anode modeling a pump may have parameters such as density, shaft speed,volume flow ratio, hydraulic power, etc. If the derivatives aredifferentiable, then backpropagation 615 can be used to determine thepartial derivatives. Backpropagation finds the derivative of the error(given by the cost function) for the parameters in the neural network,that is, backpropagation computes the gradient of the cost function withrespect to the parameters within the network. More specifically,backpropagation 615 calculates the derivative between the cost functionand parameters by using the chain rule from the last neurons calculatedduring the feedforward propagation through the internal neurons, to thefirst neurons calculated—a backward pass; that is, taking the gradientof the thermodynamic model backward in relation to the cost. In someembodiments, backpropagation will be performed by automaticdifferentiation 620. According to Wikipedia, “automatic differentiationis accomplished by augmenting the algebra of real numbers and obtaininga new arithmetic. An additional component is added to every number torepresent the derivative of a function at the number, and all arithmeticoperators are extended for the augmented algebra.” Other methods may beused to determine the parameter partial derivatives. These includeParticle Swarm and SOMA ((Self-Organizing Migrating Algorithm), etc. Thebackpropagation may work on a negative gradient of the cost function, asthe negative gradient points in the direction of smaller values.

After the partial derivatives are determined, the control sequence isoptimized 625 to lower the value of the cost function with respect tothe inputs. This process is repeated incrementally. Many differentoptimizers may be used, which can be roughly grouped into 1) gradientdescent methods 630 and 2) other methods 635. Among the gradient descentmethods 630 are standard gradient descent, stochastic gradient descent,and mini-batch gradient descent. Among the other methods 635 areMomentum, Adagrad, AdaDelta, ADAM (adaptive movement estimation), and soon. Once a new sequence is determined, the neural network is run again420.

FIG. 7 depicts a physical system 700 whose behavior can be determined byusing a neural network. This physical system 700 comprises a simpleheating system comprising a pump 725, a boiler 740, and a heating coil750 that produces hot air. The pump itself comprises a control 705 tosend a signal to turn the pump on to a relay 710, which then sends powerto a motor 720, that drives a pump 725. The pump sends water to a boiler740, which is likewise turned on by a control 730-relay 735 power 745system. The boiler then sends hot water to a heating coil, whichtransforms the hot water into hot air.

FIG. 8 depicts a heterogenous neural network 800 that may be used tomodel behaviors of the physical system of FIG. 7 . Nodes are placed inlocations with reference to the physical equipment behavior, such thatthe control node 805 is connected to relay node 810, the relay node isconnected to Power node 815. Relay node 810 is also connected to motornode 820 and pump node 825. When the control node 805 receives an inputto turn on, that information is relayed through the relay node, whichsignals the power node to turn on and the motor node to turn on. These,in turn signal the pump node to turn on. The power node may, forexample, send a voltage signal to the relay node, which may pass it onto the motor node. An activation function of the motor node may haveassociated with it a series of equations that take the signal from therelay node and turn it into mechanical rotation for the pump node 825 touse. The pump node 825 may also have a water input 885 with its ownproperties. Similarly, the control node 830, when input with an “on,” orsome other method to indicate an on action, will turn on the boiler node840 through passing on an “on” 855 to a relay node 835, which then turnson the power node 840 through variables sent through edge 860. Powernode 845 then passes electricity along edge 865 through the relay node835 edge 875 to the boiler node 840 which then, e.g., uses variablesfrom the pump node 825 and its own activation function equations thatmodel its physics properties to do the model equivalent of heatingwater. This, in turn, passes variables that heats up the heating coilnode 850. Heating coil node 850 intakes air values along edge 870 andproduces hot air values 880. The values 880 may be the simulated demandcurve 440 for this model. In some embodiments, this system would producea neural network that used two control sequences, one for control node805, and one for control node 830. It would produce one demand curve,the output from the heating coil node 850.

In some implementations, some nodes within a neural network have manyvariables that are passed among the nodes, and have different(heterogenous) activation functions. For example, an exemplary boileractivation function may describe, using equations, the activation of aboiler, e.g., boiler node 840. This may be, in whole or in part:inputPower=inputVoltage*inputCurrent; PLR=inputPower/Nominal power;Resistance Resistance=f(Nominal pressure drop, Nominal flow rate);Efficiency=f(Efficiency coefficients, PLR, nominal temperature);Power=f(PLR, Efficiency, Full load efficiency, Capacity);specificEnthalpy=f(input specificEnthalpy, Power, fluid flow rate);Pressure drop=f(Flow, resistance); Pressure=Pressure−Pressure drop, andso forth.

Exemplary weight values in a neural network that might be used asvariables in a activation node for a boiler may be: Nominal temperature;Nominal power; Full load efficiency; Nominal pressure drop; Nominal flowrate; inputPower=inputVoltage*inputCurrent; PLR=inputPower/Nominalpower. These variables may arrive at the node through an edge fromanother node, or as an input. One node may send multiple variables toanother node.

Exemplary equations to describe a pump that are used as an activationfunction in a node, e.g., pump node 825 may be: Volume flowrate=f(qFlow, density); Volume flow rate ratio=Volume flow rate/Maxvolume flow rate; Shaft speed ratio=qAngularVelocity/Max shaft speed;Pressure head=pressure curve (Volume flow rate, shaft speed ratio, andso forth.

Exemplary weight values in a neural network that might be used asvariables in an activation node for e.g., a pump may be: PropertiesPressure curve points; Power curve points, Efficiency curve points, Maxvolume flow rate, Max pressure head, Max shaft speed, and so forth.

IV. Exemplary System Embodiment

FIG. 9 depicts one system 900 for control sequence generation. Thesystem may include a computer environment 100. The system includes atime series control sequence 905 that comprises instructions to controla controllable device over time. Initially, the values of the controlsequence may be random, may be a time series control sequence fromanother similar model run, etc. A target demand curve 910 that comprisesamount of state in a location over time is also included. As describedwith reference to FIG. 5 , a time series control sequence 505 is used asinput. The pictured control sequence describes a piece of equipmentturning on 525 and off 530 over time. Different types of equipment mayhave intermediate values and other sorts of values that are alsoindicated in a control sequence. A target demand curve 910 is alsoincluded. A target demand curve 910 is the ground truth value that themodel is trying to match. This demand curve is the amount of state overtime that should be in a space associated with the model, a spaceassociated with a portion of the model (sometimes called zones), etc.

A thermodynamic model 915 is used that models the equipment that is tobe used (or is being used) in a space. Examples of such models are shownwith reference to FIG. 7 and FIG. 8 . The model is a thermodynamic modelbecause it incorporates thermodynamic qualities of objects in the modelby way of the activation functions. The thermodynamic model may be aheterogenous physics network; heterogenous as the activation functionsof the nodes may be different, as different resources behavedifferently. This may include nodes that are thermodynamicrepresentations of equipment or portions of equipment. They may bethermodynamic representations by being associated with equations thatmodel thermodynamic behavior of the equipment/portions of equipment.

A computing engine 920 runs the thermodynamic model using the timeseries control sequence as input, and outputs a simulated demand curve.With reference to FIGS. 5 and 8 , in some embodiments, a computingengine 920 runs the thermodynamic model by inputting a control sequenceover time 505, and running the values through the model, eg., accordingto the arrows shown in FIG. 8 . The model then outputs a demand curve880. A cost function determiner 925 compares difference between thesimulated demand curve and the target demand curve producing a cost.This cost is a performance metric describes how close the simulateddemand curve is to the target demand curve.

A new model input determiner 930 determines a new time series controlsequence that should give an output demand curve with a lower cost, thatis closer to the target demand curve. The new model input determiner 930may use a backpropagator 935 to help determine the new time seriescontrol sequence. This backpropagator may use an automaticdifferentiator 940 to determine gradients of the various variables(weights, values, etc.) with the thermodynamic model. This may producethe negative gradient of the cost function for the weights and/orvariables used in the cost functions; examples of which can be seenabove. Once the negative gradients are determined, then an optimizer 945can use the negative gradients to incrementally change the time seriescontrol sequence such that the thermodynamic model output more closelyapproaches the target demand curve 910. As described with reference toFIG. 6 , the optimizer may optimize inputs use gradient methods 630 orother methods. An iterator 950 runs the thermodynamic model 915 usingthe computing engine 920. It then uses the cost function determiner todetermine how close the simulated demand curve computed by the computingengine is to the target demand curve. If a goal function has been met,e.g., if the two curves are close enough, if a certain number ofiterations have been run, if the model has run for a certain time, etc.,then the iterator quits, and the last determined time series controlsequence is considered the control sequence shown in FIG. 3 at 310 thatis output from the larger model. If a goal function has not been met,then the iterator runs the thermodynamic model 915 using the computingengine 920 until the cost function determiner determines that the goalstate has been met. Once a control sequence has been generated, it maybe used to control equipment that was modeled.

Cost functions can minimize many different aspects of a thermodynamicmodel by which variables are viewed when calculating the cost. Forexample a cost function determiner may comprise a cost function whichlooks at variables within the neural network to minimize short cyclingfrequency by calculating a cost based on, e.g., length of cycles. Othercost functions may minimize a next time series control sequence ofoperation error, a next time series output path error, an energy cost, astate cost, a frequency of sequence changes, a equipment life cost, acomfort value, etc.

FIG. 10 discloses a node system 1000 that has multiple inputs. A nodemay be affected by other forces not directly built into thethermodynamic model. For example, the value of a solar panel node 1015may depend, at least partly, on the weather 1005, the orientation, thetime of year, etc. These forces may be included in the thermodynamicmodel as a time series curve input 1010 that models the outsideinfluence as a state curve with values over a series of time steps.These values may then be incorporated into a node 1015 as a direct input1020, as feed forward value from one node to another 1025, orincorporated in a different method.

In view of the many possible embodiments to which the principles of thedisclosed invention may be applied, it should be recognized that theillustrated embodiments are only preferred examples of the invention andshould not be taken as limiting the scope of the invention. Rather, thescope of the invention is defined by the following claims. We thereforeclaim as our invention all that comes within the scope and spirit ofthese claims.

We claim:
 1. A device for control sequence generation, comprising: aprocessor in communication with a memory, the processor being configuredto: retrieve from a memory associated with the processor, a neuralnetwork which models a plurality of controlled devices; retrieve fromthe memory associated with the processor, a desired demand curve;receive a simulated control sequence that comprises instructions tocontrol the plurality of controlled devices over time for a controlledbuilding zone; run the neural network using the simulated controlsequence or a next simulated control sequence as input and receiving asimulated demand curve as output, wherein the simulated demand curve isa time series of zone energy inputs; compute a cost function using thesimulated demand curve and the desired demand curve, wherein computingthe cost function comprises determining a difference between the desireddemand curve and the simulated demand curve and wherein computing thecost function further comprises performing automatic differentiationbackward through the neural network producing the next simulated controlsequence; iteratively execute the running computing, and using stepsuntil a goal state is reached; determine that the next simulated controlsequence is a generated control sequence upon the goal state beingreached; and use at least a portion of the generated control sequence torun at least one of the plurality of controlled devices to adjust anenvironmental state of the controlled building zone.
 2. The device ofclaim 1, wherein the cost function comprises a next time series controlsequence of operation error, a next time series output path error, anenergy cost, a state cost, a short cycling frequency, a frequency ofsequence changes, a equipment life cost, or a comfort value.
 3. Thedevice of claim 1, wherein the neural network is a heterogenous physicsnetwork.
 4. The device of claim 3, wherein the heterogenous physicsnetwork comprises nodes that comprise equations that model thermodynamicbehavior of equipment.
 5. The device of claim 4, wherein the nodes arelaid out with reference to the plurality of controlled devices.
 6. Thedevice of claim 5, wherein at least one node has multiple variablesassociated with it.
 7. The device of claim 6, wherein at least two nodeshave different activation functions.
 8. The device of claim 7, whereinat least one activation function comprises multiple equations.
 9. Thedevice of claim 1, wherein a node within the neural network accepts atime series curve representing weather as input.
 10. The device of claim1, wherein the plurality of controllable devices comprise HVAC devices.11. A method executed by a processor of control sequence generation,comprising: retrieving from a memory associated with the processor, aneural network of which models a plurality of controllable devices;retrieving from the memory associated with the processor, a desireddemand curve; receiving a simulated control sequence that comprisesinstructions to control the plurality of controllable devices over timefor a building zone; running the neural network using the simulatedcontrol sequence or a next simulated control sequence as input andreceiving a simulated demand curve as output, wherein the simulateddemand curve is a time series of zone energy inputs; computing a costfunction using the simulated demand curve and the desired demand curve,wherein computing the cost function comprises determining a differencebetween the desired demand curve and the simulated demand curve andwherein computing the cost function further comprises performingautomatic differentiation backward through the neural network producingthe next simulated control sequence; iteratively executing the running,computing, and using steps until a goal state is reached; determiningthat the next simulated control sequence is a generated control sequenceupon the goal state being reached; and using at least a portion of thegenerated control sequence to run at least one of the plurality ofcontrollable devices to adjust an environmental state of the controlledbuilding zone.
 12. The method of claim 11, wherein the goal statecomprises the cost function being minimized, the neural network runningfor a specific time, or the neural network running a specific number ofiterations.
 13. The method of claim 11, wherein the neural networkcomprises multiple nodes, at least two of the multiple nodes havingdifferent activation functions, and wherein at least one activationfunction has multiple variables that are passed to a different node. 14.The method of claim 11, wherein the plurality of controllable devicescomprise HVAC devices.
 15. The method of claim 11, wherein the neuralnetwork comprises nodes that comprise activation functions that modelthermodynamic behavior of equipment.
 16. The method of claim 14, whereinthe nodes are laid out with reference to connections between theplurality of controllable devices.
 17. A non-transitorycomputer-readable storage medium configured with instructions executableby processor for creation of a desired control sequence from a desireddemand curve, the non-transitory computer-readable storage mediumcomprising: instructions for retrieving from a memory associated withthe processor, a neural network which models a plurality of controllabledevices; instructions for retrieving from the memory associated with theprocessor, a desired demand curve; instructions for receiving asimulated control sequence that comprises instructions to control theplurality of controllable devices over time for a controllable buildingzone; instructions for running the neural network using the simulatedcontrol sequence or a next simulated control sequence as input andreceiving a simulated demand curve as output, wherein the simulateddemand curve is a time series of zone energy inputs; instructions forcomputing a cost function using the simulated demand curve and thedesired demand curve, wherein computing the cost function comprisesdetermining a difference between the desired demand curve and thesimulated demand curve and wherein computing the cost function furthercomprises performing automatic differentiation backward through theneural network producing the next simulated control sequence;instructions for iteratively executing the running, computing, and usingsteps until a goal state is reached; instructions for determining thatthe next simulated control sequence is a desired time series controlsequence upon the goal state being reached; and instructions for usingat least a portion of the desired time series control sequence to run atleast one of the plurality of controllable devices to adjust anenvironmental state of the controlled building zone.
 18. Thenon-transitory computer-readable storage medium of claim 17, wherein theneural network comprises multiple nodes, at least two of the multiplenodes having different activation functions, and wherein at least oneactivation function has multiple variables that are passed to adifferent node.
 19. The non-transitory computer-readable storage mediumof claim 18, wherein each of at least two nodes in the multiple nodeslocation in the neural network correspond to each of at least two of thecontrollable devices.
 20. The non-transitory computer-readable storagemedium of claim 19, further comprising the each of at least two nodesare arranged in the neural network equivalently to an arrangement ofconnections of the each of the controllable devices.